-
Notifications
You must be signed in to change notification settings - Fork 30k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multiple fixes for async iterators #23901
Conversation
@@ -150,7 +153,7 @@ const createReadableStreamAsyncIterator = (stream) => { | |||
}); | |||
|
|||
finished(stream, (err) => { | |||
if (err) { | |||
if (err && err.code !== 'ERR_STREAM_PREMATURE_CLOSE') { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this the only way to fix it? I would’ve expected us to perhaps do something with end-of-stream rather than selectively ignoring errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
end-of-stream treats non-ending (destroy) streams that close early as an error condition. I think it is right in doing so.
However this does not play well with async iterators as you pointed out. We had a different logic before that did the same but was less explicit about it: I like this one better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
end-of-stream treats non-ending (destroy) streams that close early as an error condition. I think it is right in doing so.
Can you elaborate? It sounds to me that destroyed streams (with null
as error value) should be treated no differently as any other stream that ended. I.e., it's reasonable to believe a stream can be destroyed whenever.
In fact, I would say that the following test case added in #23785 should in fact finish gracefully with an { done: true, value: undefined }
rather than throwing an error:
node/test/parallel/test-stream-readable-async-iterators.js
Lines 328 to 343 in 482b56a
await (async function() { | |
console.log('.next() on destroyed stream'); | |
const readable = new Readable({ | |
read() { | |
// no-op | |
} | |
}); | |
readable.destroy(); | |
try { | |
await readable[Symbol.asyncIterator]().next(); | |
} catch (e) { | |
assert.strictEqual(e.code, 'ERR_STREAM_PREMATURE_CLOSE'); | |
} | |
})(); |
We see precedent for invalidated iterator still operating gracefully in ECMA-262 iterators:
const array = [0, 1, 2, 3];
const iterated = [];
for (const a of array) {
iterated.push(a);
if (a === 1) {
array.splice(0);
}
// loop ends immediately.
}
// iterated is [0, 1]
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First, end-of-stream is one of the most downloaded utilities on npm, and keeping the API and behavior similar has been a goal. Second, from a consumer perspective a stream that was ended or destroyed is significantly different. That could have easily been a flag as an argument. I think it errors because in the context of pipeline/pump, in fact it is an error: the pipeline did not complete correctly but was interrupted (as an example, gzipping a file).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’ll check again that test asap, it does not sound correct with this change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, as long as async iteration isn't affected it works for me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@TimothyGu the test was botched anyway, and the catch clause was never executed in this case. I've fixed it.
2566bab
to
3d41222
Compare
landed into 10.x with #23785 opted to not land in 8.x (I don't believe the async iteration work is on that release stream) |
It should not backported to 8.
Il giorno lun 26 nov 2018 alle 22:41 Myles Borins <notifications@github.com>
ha scritto:
… landed into 10.x with #23785 <#23785>
opted to not land in 8.x (I don't believe the async iteration work is on
that release stream)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#23901 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AADL41zXgv77KrIyOZbZagNzfwRotU-Aks5uzGAVgaJpZM4X8EO3>
.
|
Fixes: #23890
Fixes: #23891
Checklist
make -j4 test
(UNIX), orvcbuild test
(Windows) passes